The present invention relates to color imaging device and color imaging apparatus capable of reducing size and cost, and suppressing color moire.
Imaging devices, typically image pick-up tubes and solid-state imaging devices, are extensively used for imaging apparatuses. Particularly, single-tube or single-sensor color imaging devices used for color imaging apparatuses, have a merit that an imaging apparatus can be constructed with a single imaging device. The devices also have many other merits such as no requirement of any color separating prism causing lens size reduction, free from various multiple sensor type adjustments, typically registration, and consume low power. The devices have many contributions to the size and power consumption reduction of imaging apparatuses. Particularly, single-sensor color cameras using color CCD imaging devices which are solid-state devices, have become leading imaging apparatuses.
The above color imaging devices all obtain color information with a single light-receiving surface by color coding therein with color filters called stripes filters or mosaic filters. For example, three, i.e., R, G and B, color filters are applied in a predetermined regular array to each photo-electric converting element, thus providing a peculiar spectral sensitivity to each pixel. Thus, an image signal obtained by imaging a scene contains point-sequential color data corresponding to the predetermined color filter array. It is thus possible to take out color data by separating and taking out the signal corresponding to each color filter in compliance with the predetermined color filter array. To obtain luminance signal (or Y signal), at least three pixels (i.e., one R, one G and one B pixel) are necessary, and this means that the color imaging can be obtained with a single imaging device although the luminance resolution is sacrificed.
RGB Bayer array is one of such well-known arrays as noted above. While several arrays are well known as the Bayer array, FIG. 6 shows a typical one of such arrays. This array is obtained by sequentially arranging a plurality of two-dimensional unit arrays each of four, i.e., 2xc3x972, pixels to fully fill a plane, that is, it is a two-dimensional periodic array of four-pixel, i.e., (2xc3x972)-pixel, unit arrays.
FIG. 7 shows another example of the RGB stripes array. This array is constituted by three color filter stripes (arranged as sequential columns), that is, it is a two-dimensional periodic array of unit arrays each of three, i.e., 3xc3x971, pixels.
Both the above RGB Bayer and RGB stripes arrays use original (RGB) color filters of good color reproducibility. The Bayer array has a feature that the proportions of the R, G and B pixel numbers are set to 1:2:1, that is, an increased density of G pixels which have great contribution to the luminance signal is provided, thus providing an increased luminance resolution. In addition, since the pixels are arranged likewise in the vertical and horizontal directions, the resolutions obtainable in the two directions are alike.
The stripes array has no color coding in the vertical direction, and its luminance resolution in this direction is extremely high (i.e., as high as comparable to the monochromatic case). In addition, since the R, G and B pixel densities are the same, this array features that the color signal-to-noise ratio is good and that the color reproducibility is better than that of the Bayer array.
Although the above Bayer and stripes arrays are excellent as described above, in the usual imaging device no particular consideration is given to the securing of the dynamic range (i.e., luminance reproduction range) of imaging a scene. Therefore, imaging of a scene having a great luminance distribution range from high to low luminance readily results in white missing or blackening.
More specifically, the imaging range is not simply determined by the sole imaging device, but it also depends on the signal processing in the imaging apparatus using the imaging device. More specifically, on the high luminance side the saturation level of the imaging device is a limit, and on the low luminance side the noise level of the imaging device output assembled in the imaging apparatus is a limit. Therefore, it has been impossible to obtain an imaging range which at least exceeds the above range.
A usual imaging device used for constructing an imaging apparatus has a photoelectric conversion characteristic as shown in the graph of FIG. 8. In the graph, the ordinate is taken for the logarithm of the signal level, and the abscissa is taken for the logarithm of the incident light intensity. In the graph, UL represents a high luminance side limit level, and LL represents a low luminance side limit level. The level UL substantially corresponds to the saturation level of the imaging device. The level LL, on the other hand, is not the noise level NL itself, but is determined as a signal level having such a predetermined limit signal-to-noise ratio as to withstand appreciation even in coexistence with noise. The range between the levels UL and LL is the effective luminance range, that is, the difference (UL-LL) between these ranges (on the logarithmic axis) is the imaging range.
The imaging range is in many cases about 5 to 6 EV (30 to 36 dB) although it depends on the design and manufacture of the imaging apparatus, and its further improvement has been desired. However, it has been difficult to further improve the range because of limitations imposed on the improvement of the saturation level of the imaging device and the noise level.
Now, among a variety of color coding patterns, which have been proposed and used in practice as the filter array, are 3-original-color filters such as RGB stripes filters and Bayer type RGB mosaic filters (including various varieties) and complementary color filters such as 4-color, e.g., YeMgCy stripes and YeMgCyw and YeMgCyG, mosaic filters.
The present invention points out essential problems, which are inherent in the electronic structures of the color imaging device (such as picture tube, solid-state imaging device, CCD and other types) and the various kinds of color coding (such as original colors and complementary colors or three colors and four colors), and show means for solving the problems. In the following description, unless particularly noted otherwise, only examples are considered.
Among the prior art color coding arrays, an example of Bayer type RGB arrays will now be described with reference to FIGS. 13(A) and 13(B). As shown in FIG. 13(A), the Bayer type RGB array is constituted by a plurality of unit arrays each of four, i.e., (2xc3x972), pixels. As shown in FIG. 13(B), these unit arrays are sequentially arranged to fill a plane. This array has a feature that the proportions of the R, G and B pixel numbers are set to 1:2:1, that is, an increased density of G pixels which have great contribution to luminance signal is provided for increasing luminance resolution. In addition, since the pixels are arranged likewise in the vertical and horizontal directions, the resolutions obtainable in the two directions are alike, which is different from the stripe filter. The array shown in FIG. 13(B) is constituted by 64, i.e., (8xc3x978), pixels.
However, since the Bayer type array uses a regular array as described above, it poses a significant problem causing false resolution image or so-called color moire due to space sampling based on its array. An intrinsically colorless, i.e., monochromatic, scene will now be considered, which happens to contain a scene portion having a luminance pattern (i.e., white-and-black pattern) of the same period as the period of the array. Assuming that an RG row as one horizontal line of the scene is such that R represents white color and G represents black color, the scene causes the output of a signal, which is equivalent to a signal obtainable from a red scene free from luminance changes, that is, an output of a color which is not intrinsically present is generated. Due to such stripes-like iteral pattern, the false color signal or color moire is generated in a low frequency band by so-called frequency folding-back (or areaging), and it can not therefore be removed even by a subsequent electric filtering process or the like including color band suppression.
Accordingly, the optical system of the prior art single sensor color imaging apparatus essentially includes crystal or like optical low-pass filter for ensuring the image quality. However, such an optical low pass filter imposes a great restriction on the size and cost reduction, and also it nevertheless can not completely eliminate image quality deterioration due to the residual color moire.
Aside from this problem, in the prior art imaging apparatus no particular consideration is given to the securing of the dynamic range (i.e., luminance reproduction range) of imaging a scene. Therefore, a problem has been posed that imaging of a scene having wide luminance distribution range from high to low luminance readily results in white missing or blackening.
More specifically, the imaging range is not simply determined by the sole imaging device, but it also depends on the signal processing in the imaging apparatus using the imaging device. More specifically, on the high luminance side the saturation level of the imaging device is a limit, and on the low luminance side the noise level of the imaging device output assembled in the imaging apparatus is a limit. Therefore, it has been impossible to obtain an imaging range which at least exceeds the above range. A usual imaging device used for constructing an imaging apparatus has a photoelectric conversion characteristic as shown in the graph of FIG. 8.
In the graph, the ordinate is taken for the logarithm of the signal level, and the abscissa is taken for the logarithm of the incident light intensity. In the graph, UL represents a high luminance side limit level, and LL represents a low luminance side limit level. The level UL substantially corresponds to the saturation level of the imaging device. The level LL, on the other hand, is not the noise level itself, but is determined as a signal level having such a predetermined limit signal-to-noise ratio as to withstand appreciation even in coexistence with noise. The range between the levels UL and LL is the effective luminance range, that is, the difference (UL-LL) between these ranges (on the logarithmic axis) is the imaging range.
The imaging dynamic range is in many cases about 5 to 6 EV (30 to 36 dB) although it depends on the design and manufacture of the imaging apparatus, and its further improvement has been desired. However, it has been difficult to further improve the range because of limitations imposed on the improvement of the saturation level of the imaging device and the noise level.
An object of the present invention is to solve the above problem by the provision of a high image quality imaging apparatus, which features a revolutionally improved imaging range while using an imaging device comparable to the prior art device and basically having the advantages of the prior art stripes array, and an imaging device suited for the same.
Another object of the present invention is to provide an imaging apparatus, which is based on a novel multiple pixel unit array imaging system obtained as a specific means for solving the above problem and having a wide scope of applications, and an imaging device suited for the same.
Other object of the present invention is to improve the imaging dynamic range in the prior art color imaging apparatuses, and to provide an imaging apparatus, which is free from color moire generation even with a scene having a periodic luminance change, permits greatly improving the imaging dynamic range, size and cost reduction and high image quality color imaging, and also a color imaging device permitting the same to be obtained.
According to an aspect of the present invention, there is provided an imaging device having an imaging pixel array formed as a two-dimensional periodic array of N-pixel (N being 5 or a greater natural number) arrays as unit arrays.
According to this invention, it is possible to realize an imaging device is obtainable, which permits various imaging quality improvements that are not obtainable with prior art unit arrays of 4 of less pixels.
Here, N may be 6, and of six, i.e., 1-st to 6-th, pixels constituting each unit array, the 1-st to 3-rd pixels have different color characteristics (i.e., relative spectral sensitivity characteristics), and the 4-th to 6-th pixels are different in the sensitivity (i.e., absolute sensitivity) from and the same in the color characteristic as the 1-st to 3-rd pixels, respectively.
According to this invention, with 6-pixel unit arrays it is possible to very easily realize applications to two-density type unit arrays based on the prior art 3-pixel unit arrays, and it is possible to realize an imaging device, which permits greatly enlarging the imaging range while basically having the features of the prior art 3-pixel unit arrays.
The three different color characteristics are three original colors R, G and B to be additively mixed.
According to this invention, it is possible to realize an imaging device, which permits greatly enlarging the imaging range while basically having the features of the prior art RGB stripes array.
According to another aspect of the present invention, there is provided an imaging apparatus comprising an imaging device having an imaging pixel array formed as a two-dimensional periodic array of unit arrays each constituted by six, i.e., 1-st to 6-th, pixels, the 1-st to 3-rd pixels having different color characteristics (i.e., relative spectral sensitivity characteristics), the 4-th to 6-th pixels being different in the sensitivity (i.e., absolute sensitivity) from and the same in the color characteristic as the 1-st to 3-rd pixels, respectively, and an image signal generating means for generating an image signal having a predetermined form according to 1-st to 6-th pixel data signals obtained in correspondence to the 1-st to 6th pixels in the imaging device, wherein: the different sensitivities of the 1-st and 4-th pixels, as well as the 2-nd and 5-th pixels and the 3-rd and 6-th pixels in the imaging device are set such that the effective luminance range of the 1-st, as well as 2-nd and 3-rd pixel data signals have common range with the 4-th, as well as 5-th and 6-th pixel data signals, respectively.
According to this invention, it is possible to realize a high image quality imaging apparatus, which has a very wide imaging range while providing entirely the same performance as that of an imaging apparatus, which uses a prior art imaging device using 3-pixel unit arrays (for instance an RGB stripes array), for medium luminance range scenes.
The image signal generating means includes a level compensating means for compensating a signal level difference between the 1-st, as well as 2-nd and 3-rd, pixel data signal and the 4-th, as well as 5-th and 6-th, pixel data signal with respect to the same brightness scene.
According to this invention, it is possible to realize a practical imaging apparatus, which can generate image signal by compensating for a pixel sensitivity difference produced in high image quality imaging using the imaging device according to the present invention.
The image signal generating means includes a pixel data extrapolating means for executing, in the case of effective luminance range deviation while data signals of neighbor pixels of different sensitivities and the same color are not deviating the range in the pixel data signal processing, extrapolation with the data signals of the different sensitivity, same color neighbor pixels.
According to this invention, it is possible to realize an imaging apparatus, which has entirely the same performance as an imaging apparatus using a prior art imaging device using 3-pixel unit array (for instance an RGB stripes array) for scenes with only a high or low portion of the luminance range so long as the scenes meet predetermined conditions.
According other aspect of the present invention, there is provided a color imaging device having a pixel group of a plurality of pixels constituted by photoelectric converting elements, wherein the pixels are arranged in a 6-color random color coding array meeting an array prescription that the pixels are arranged in a 6-color random color coding array meeting a requirement that the pixels adjacent to the four sides and the four corners of a pixel under attention includes pixels of five different colors other than the color of the pixel under attention at least one pixel each.
According to this invention, it is possible to realize a color imaging device, which permits imaging a scene with a non-periodic, i.e., random, color coding array and, since it adopts a 6-color random array meeting a requirement that the pixels adjacent to the four sides and the four corners of a pixel under attention includes pixels of five different colors other than the color of a pixel under attention at least one pixel each, is free from color moire generation, can ensure a resolution at a predetermined high level or above over the entire imaging range and can improve the image quality performance and functions.
The 6-color random color coding array has six colors with two thereof constituting each of three original colors while being different in sensitivity.
According to this invention, since the 6-color random array has six colors with two thereof constituting each of the original colors while being different in sensitivity, it is possible to greatly improve the dynamic range of imaging.
According to still other aspect of the present invention, there is provided a color imaging apparatus comprising a color imaging device according to above two invention, and a color separating means for executing a color separating process on output signal of the color imaging device on the basis of the random color coding array of the color imaging device.
According to this invention, it is possible to provide a color imaging apparatus capable of executing reliable color separation according to color coding array data of a color imaging device, which is free from color moire generation, has a sensitivity at a predetermined high level or above, can improve the image quality performance and functions and can greatly improve the dynamic range of imaging.
The color imaging apparatus of the above invention further comprises a memory means for storing array data concerning the random color coding array of the color imaging device, the array data being stored for the execution of the color separating process in the color separating means.
According to this invention, with the provision of the memory means for storing array data concerning the random color coding array of the color imaging device, it is possible to permit ready and reliable color separation according to the random color coding array data.
The memory means is constituted by a masked ROM.
According to this invention, with the provision of the masked ROM as memory means for storing the color coding array data, it is possible to manufacture the memory means, and hence the color imaging apparatus, at low cost and by mass reduction.
The memory means is constituted by an EEPROM.
According to this invention, with the provision of the EEPROM as memory means for storing the color coding array data, it is possible to readily cope with color separating processes of color imaging devices having different color coding arrays.
Other objects and features will be clarified from the following description with reference to attached drawings.